feat: Issue Deduplication #11
Conversation
Unused types (1)
|
|
@0x4007 I have tried to make a few examples, let me know if have more or want to test more. Screen.Recording.2024-09-13.at.7.08.57.PM-1.mov |
src/handlers/issue-deduplication.ts
Outdated
| import { IssueSimilaritySearchResult } from "../adapters/supabase/helpers/issues"; | ||
| import { Context } from "../types"; | ||
| const MATCH_THRESHOLD = 0.95; | ||
| const WARNING_THRESHOLD = 0.5; |
There was a problem hiding this comment.
A cosine similarity of 0.75 appears quite close for identifying similar issues. I tested this with a few examples and noticed some potential errors with the samples. Typically, for similar issues, the similarity was either above 75% and aligned with 95% category or around 60%. Therefore, I experimented with a 50% threshold, which seemed to work well.
Added They will display the cosine similarity in percentage after each issue in the list.
I manually calculated and created test cases using embeddings and found their cosine similarity values. |
|
Can you link your issue where you tested so we can see the results? |
95%: 50%: I have deployed the plugin at Plugin Link, if you wish to try it. The issues test values Link |
|
Okay it seems like you aren't following the spec again. Needs to list the similar results on every scenario. Do 75% and 95% as a default. |
Fixed that, it now returns the similar issue in both
Warning Threshold is 75% now. 95%: 75%: |
Doesn't look like it in the first one |
That's the first issue of that type, so its expected to not have similar issues. Two, should be the first time a similar issue, is found with similarity more than 95%. So, the first issue would not satisfy the any of the match conditions. The third issue does not have any similar issues to that, so it wouldn't have any message. |
Resolves #6